Earley- or Tomita-style algorithms, are two-fold. First, 
at the syntactic level, any kind of chart parsing 
algorithm faces combinatorial problems with non- 
contiguous grammar specifications (accounting for 
discontinuous language structures) and, in particular, 
extra- and ungrammatical language input (cf., e.g., 
Magerman & Weir (1992) for probabilistic and Lee 
et al. (1995) for symbolic heuristics to cope with that 
problem). Thus, under realistic conditions, these tech- 
niques loose a lot of their theoretical appeal and com- 
pete with other approaches merely on the basis of per- 
formance measurements. Second, including seman- 
tic considerations, even if we assume efficient syntac- 
tic processing for the sake of argument, the question 
arises how semantic interpretations can be processed 
in an incremental, comparably efficient way. Though 
experiments have been run with packing feature struc- 
tures and interleaving syntactic and semantic analyses 
(Dowding et al., 1994), or with the intentional under- 
specification of logical forms (leaving scope ambigui- 
ties of quantifiers and negations underdetermined; cf., 
e.g., Hobbs (1983) or Reyle (1995)), no conclusive ev- 
idences have been generated so far in favor of a gen- 
eral method for efficient, online semantic interpreta- 
tion. As we are faced, however, with the problem to 
work out text interpretations incrementally and within 
reasonable resource bounds, we opt for a methodol- 
ogy that constrains the amount of ambiguous struc- 
tures right at the source. Hence, the incompleteness of 
the algorithm trades theoretical purism for feasibility 
of realistic NLP. 

5 Conclusions 

We have presented a restricted approach to paral- 
lelism for object-oriented lexicalized parsing. Given 
the complex control structure requirements of a real- 
istic text understanding system (integrated, incremen- 
tal, robust processing), we argued for a unifying ap- 
proach in which declarative grammar constraints are 
lexically encoded and procedural knowledge can be 
specified by distinguished lexicalized conmiunication 
primitives (viz.. a message passing protocol). This led 
us to the description of a concurrent parsing algorithm 
which is characterized by a depth-first, robust, yet in- 
complete analysis of textual input. We also argued in 
favor of incompleteness in order to break the text pars- 
ing complexity barrier. As a consequence, we do not 
only supply an efficient parsing procedure but also one 
that is effective in the sense that it guarantees the gen- 
eration of conceptual representations of the content of 
the text under feasible resource demands. 

Acknowledgments. P. Neuhaus is supported by a 
grant from DFG within the Freiburg University Grad- 
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Neuhaus & Hahn (1996)). In the scenario we have dis- 
cussed, the ReceiptHandler eventually will detect 
the success and the termination of the search head pro- 
tocol. Next, the new ContainerActor will be sent 
an analyzeWithContext : message to continue 
the parsing process. 

3.4 Preliminary Experimental Evaluation 

The efficiency gain that results from the parser design 
introduced in the previous sections can empirically 
be demonstrated by a comparison of the ParseTalk 
system (abbreviated as "PT" below) with a standard 
active chart parser^ (abbreviated as "CP"). Since the 
chart parser is implemented in Smalltalk, while the 
ParseTalk system is implemented in Actalk (Briot, 
1989) — a language which simulates the parallel exe- 
cution of actors — , a direct comparison of run times is 
not reasonable (though even at that level of consider- 
ation the ParseTalk system outperforms the chart 
parser). We therefore compare, at a more abstract 
computation level, the number of method executions 
given exactly the same dependency grammar^. The 
computationally most expensive methods we consider 
are syntaxCheck and conceptCheck (cf. Sec- 
tion 3.1). Especially the latter consumes large compu- 
tational resources, since for each interpretation vari- 
ant a knowledge base context has to be built and con- 
ceptual consistency must be checked. Therefore, it 
is only considered when the syntactic criteria are ful- 
filled. The number of calls to these methods for a sam- 
ple of 13 randomly chosen, increasingly complex sen- 
tences from the information technology domain test 
library is given by Fig. 4 ("CP.syn" and "PT.syn") and 
Fig. 5 ("CPcon" and "PT.con"). A reduction by a fac- 
tor of four to five in the (unweighted) average case can 
be observed applying the ParseTalk strategy. 

Furthermore, the ParseTalk parser, by design, is 
able to cope with discontinuities stemming from un- 
or extragrammatical input. The performance of a re- 
vised version of the chart parser which also handles 
these cases is given as "CP.disc.syn/con" in the fig- 
ures. The missing value for sentence 10 results from 
the chart parser crashing on this input because of space 
restrictions of the run-time system (the experiments 
were conducted on a SPARCstation 10 with 64 MB 
of main memory). The average reduction in compar- 

'Winograd' s ( 1 983) chart parser was adapted to parsing a 
dependency grammar. No packing or structure-sharing tech- 
niques could be used, since semantic interpretation occurs 
online, thus requiring continuous referential instantiation of 
structural linguistic items (cf. also Section 4). 

*This can only serve as a rough estimate, since it does not 
take into account the exploitation of ParseTalk' s concur- 
rency. Furthermore, the chart parser performs an extremely 
resource-intensive subsumption checking method unneces- 
sary in the ParseTalk system. 
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Figure 4: Calls to syntaxCheck 
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Figure 5: Calls to conceptCheck 

ison with the extended version of the chart parser is 
about six to nine. 

4 Related Work 

Research on object-oriented natural language pars- 
ing actually started with the work of Small & Rieger 
(1982) on word experts. Based on a conceptual pars- 
ing model, this approach took a radical position on 
full lexicalization and communication based on a strict 
message protocol. Major drawbacks concerned an 
overstatement of the role of lexical idiosyncrasies and 
the lack of grammatical abstraction and formalization. 
Preserving the strengths of this approach (lexicalized 
control), but at the same time reconciling it with cur- 
rent standards of lexicalized grammar specification, 
the ParseTalk system can be considered a unifying 
approach which combines procedural and declarative 
specifications at the grammar level in a formally disci- 
plined way. This also distinguishes our approach from 
another major stream of object-oriented natural lan- 
guage parsing which is almost entirely concerned with 
implementational aspects of object-oriented program- 
ming, e.g., Habert (1991), Lin (1993) or Yonezawa & 
Ohsawa (1994). 

The reasons why we diverge from conventional 
parsing methodologies, e.g., chart parsing based on 
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Figure 1: Protocol for Word Actor Initialization 



a ReceiptHandler is instantiated by a syn- 
chronous message, intended to detect the partial ter- 
mination of the subsequently started search proto- 
col. The perf ormSearchHead message triggers 
(via perf ormSearchHeadTo : messages) asyn- 
chronous searchHeadFor : messages to be for- 
warded by each receiving PhraseActor to its right- 
most WordActor. From this origin, the search mes- 
sage can be distributed to all word actors at the right 
"rim" of the dependency tree by simply forwarding 
it to the respective heads. After forwarding^, each 
searchHead message evokes the check of syntactic 
and semantic restrictions by the corresponding meth- 
ods. These restrictions must be met in order to estab- 
lish a dependency relation between the receiving word 
actor and the sender of the message. Provided that 
these constraints are fulfilled, an attach : message 
is sent to the encapsulating PhraseActor. Before 
the new composite phrase can be built, the address of 
the next container actor must be determined. Accord- 
ingly, the getNextContainer message either re- 
turns this address directly or, if it is not available yet, it 
will create the next container actor first (the actual cre- 
ation protocol is not shown). The newin : message 
subsequently creates a new PhraseActor that will 
encapsulate the word actors of the new phrase. No- 
tice, that several attach : messages can be received 
by a phrase, because the searchHead messages are 
evaluated in parallel by its word actors. 

In order to actually build the new phrase a 
copyAndAttach : message is sent-'. Fig. 3 depicts 
the copying of the governing and modifying phrases 

^Since forwarded messages are sent asynchronously the 
processing of the searchHeadFor: message takes place concur- 
rently at the forwarding sender and the respective receivers. 

'As an alternative to the immediate establishment of a 
dependency relation, a headFound message can be returned 
to enable the subsequent selection of preferred attachments 
(cf. Neuhaus & Hahn (1996) for such a protocol extension). 
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Figure 2: Protocol for the Search for a Head 
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Figure 3: Protocol for Establishing a Dependencies 

into the new PhraseActor by copyHeadFor: 
and copyModFor : messages, respectively — copy- 
ing enables alternative attachments in the concur- 
rent system, i.e., no destructive operations are carried 
out. Note that in contrast to the searchHead mes- 
sage the PhraseActor forwards the copy message 
to its root actor from where it is distributed in the 
tree. The dependency relation connecting the copied 
phrases (indicated by the bold edge in the newly built 
PhraseActor) is created by the establish: 
message. Since word actors hold information (such 
as features or coverage) locally, updates are necessary 
to propagate the effects of the creation of the relation. 
After updating information at relevant word actors in 
the resultant tree, successful termination of the search 
message is signalled to the ReceiptHandler. If 
none of the receipts signals success to the handler, 
the search head protocol will be followed by modifier 
search or backtracking"* protocols not shown here (cf . 

"^Thus synchronization of protocols enables word-wise 
scanning, backtracking, etc. This avoids severe problems 
usually encountered in parsers with unrestricted parallelism. 



case) number of spuriously ambiguous analyses, or 
the global operation of subsequent duplicate elimina- 
tion. This led us to restrain from unbounded paral- 
lelism and, rather, guarantee confluent behavior by the 
design of the parsing algorithm. 

Confluency. In the first prototype, we enforced con- 
fluency by an incremental structure-building condition 
on the basis of a synchronization schema. Messages 
were forwarded strictly from right to left wandering 
through the preceding context rather than being broad- 
casted. Partial structures were organized such that a 
message which could be successfully processed at a 
larger structure was not forwarded to any of its con- 
stituent parts. But still, the number of ambiguities re- 
mained prohibitively large, often due to unnecessary 
partial structures with large discontinuities. For in- 
stance, any determiner preceding a noun forms a new 
structure, with the Det modifying the N. Usually, a 
contiguity restriction would filter out those structures 
given perfectly well-formed input. But such a restric- 
tion is detrimental to requirements set up in a realis- 
tic text parsing environment, in which the analysis of 
(possibly large) fractions of un- as well as extragram- 
matical input must be skipped. Furthermore, order- 
ing restrictions on dependency analyses for German 
can be formulated more transparently, if discontinu- 
ous structures are allowed. 

Depth-First Approach. These experiences led to a 
redesign of the first prototype. The parser' s forward- 
ing mechanism for search messages was further re- 
stricted to circumvent the above mentioned problem 
of erroneous discontinuous (over)analyses. In this ap- 
proach, we let phrases that constituted alternative anal- 
yses for the same part of the input text be encapsu- 
lated in a container actor. Container actors play a 
central role in controlling the parsing process, because 
they encapsulate information about the preceding con- 
tainers that hold the left context and the chronologi- 
cally previous containers, i.e., a part of the parse his- 
tory. Container actors comprising single- word phrases 
are called lexical containers. All phrases in the ac- 
tive container send a search message to the current 
context container that forwards them to its encapsu- 
lated phrases. The search messages are then asyn- 
chronously distributed to words within each phrase. If 
at least one attachment to one of these phrases is pos- 
sible, no further forwarding to containers which cover 
text positions preceding the current position will oc- 
cur. Thus, the new container composed at this stage 
will contain only those phrases that were encapsulated 
in the context container and that could be enlarged by 
attaching a phrase from the active container. 

This procedure enforces a depth-first style of pro- 
gression, leaving unconsidered many of the theoreti- 



cally possible combinations of partial analyses. Still, 
some information has to be retained in order to back- 
track after failures or to employ alternative parsing 
strategies. We encounter a trade-off between robust- 
ness, efficiency, and completeness in parsing. If we 
were to allow for unrestricted backtracking, we would 
just trade in run- time complexity for space complex- 
ity (for a more detailed discussion, cf. Neuhaus & 
Hahn (1996)). Hence, we rather restrict backtrack- 
ing to those containers in the parse history which hold 
governing phrases, while the containers with modify- 
ing phrases are immediately deleted after attachment^ 

3.3 Restricted Parallel Parsing Algorithm 

The parsing algorithm of the ParseTalk system is 
centered around the head search process of the cur- 
rently active word actor. If it fails, a modifier search 
process is triggered; if it succeeds, a new dependency 
structure is constructed combining the partial analy- 
ses. In case both of these protocols are not successful, 
containers may be skipped so that discontinuous anal- 
yses may occur. If the skipping process encounters a 
linguistically valid boundary (in the most trivial case, 
the punctation mark of the previous sentence) it stops 
and switches to a backtracking mode leading to a kind 
of roll-back of the parser invalidating the currently 
pursued analysis. In a companion paper (Neuhaus & 
Hahn, 1996), we give an integrated description of the 
various subprotocols needed for head/modifier search, 
ambiguity handling, skipping, backtracking, preferen- 
tial and predictive parsing. 

In this paper, we concentrate instead on the basic 
message passing patterns for the establishment of de- 
pendency relations, viz. the searchHead protocol, 
and its concurrency aspects. For illustration purposes 
we here introduce the protocol in a diagrammatic form 
(Figs. 1 to 3). The figures depict the main steps of 
word actor initialization, head search, and phrasal at- 
tachment. This format eases communication, while 
formal specifications based on a temporal logic frame- 
work are used when dealing with formal properties of 
the parser (cf., e.g., Schacht (1995) for a partial termi- 
nation proof of the receipt handler introduced below). 

The parser is started by an analyze: message 
directed to a ParserActor, which is responsible 
for the global administration of the parsing process 
(cf. Fig. 1). It instantiates a LexicalContainer- 
Actor that encapsulates the (potentially) ambiguous 
readings of the first word of the text, as accessed from 
the lexicon and corresponding word classes. 

Upon receiving the analy zeWithContext : 
message from the ParserActor (cf. Fig. 2), 

'Hence, the incompleteness property of our parser stems 
from the selective storage of analyses (i.e., an "incomplete 
chart" in chart terms), partially compensated by reanalysis. 



3 Object-oriented Lexical Parsing 

In this section, we introduce the ParseTalk system, 
whose specification and implementation is based on 
an object-oriented, inherently concurrent approach to 
natural language analysis. We consider constraints 
which introduce increasing restrictions on the paral- 
lel execution of the parsing task. This leads us to a 
parsing algorithm with restricted parallelism, whose 
experimental evaluation is briefly summarized. 

3.1 The ParseTalk Model 

The ParseTalk grammar we use (for a survey, cf. 
Broker et al. (1994)) is based on binary relations be- 
tween words, e.g., dependency relations between a 
head and its modifier, or textual relations between an 
anaphor and its antecedent. Restrictions on possible 
relations are attached to the words, e.g., expressed as 
valencies in the case of dependency relations, yielding 
a strictly lexicalized grammar in the sense of Schabes 
et al. (1988). The individual behavior of words is gen- 
eralized in terms of word classes which are primarily 
motivated by governability or phrasal distribution; ad- 
ditional criteria include inflection, anaphoric behavior, 
and possible modifiers. A word class specifies mor- 
phosyntactic features, valencies, and allowed order- 
ings for its instances. Further abstraction is achieved 
by organizing word classes at different levels of speci- 
ficity in terms of inheritance hierarchies. The specifi- 
cation of binary constraints already provides inherent 
means for robust analysis, as grammatical functions 
describe relations between words rather than well- 
formed constituents. Thus, ill-formed input does often 
still have an (incomplete) analysis. 

The ParseTalk parser (for a survey, cf. Neuhaus 
& Hahn (1996)) generates dependency structures for 
sentences and coherence relations at the text level of 
analysis. In order to establish, e.g., a dependency re- 
lation the syntactic and semantic constraints relating 
to the head and its prospective modifier are checked 
in tandem. Due to this close coupling of grammat- 
ical and conceptual constraints syntactically possible 
though otherwise disallowed structures are filtered out 
as early as possible. Also, the provision of con- 
ceptual entities which are incrementally generated by 
the semantic interpretation process supplies the neces- 
sary anchoring points for the continuous resolution of 
textual anaphora and ellipses (Strube & Hahn, 1995; 
Hahn et al., 1996). 

The lexical distribution of grammatical knowledge 
one finds in many lexicalized grammar formalisms 
(e.g., LTAGS (Schabes et al., 1988) or HPSG (Pollard 
& Sag, 1994)) is still constrained to declarative no- 
tions. Given that the control flow of text understand- 
ing is globally unpredictable and, also, needs to be 
purposefully adapted to critical states of the analysis 



(e.g., cases of severe extragrammaticality), we drive 
lexicalization to its limits in that we also incorporate 
procedural control knowledge at the lexical grammar 
level. The specification of lexicalized communication 
primitives allows heterogeneous and local forms of in- 
teraction among (groups of) lexical items. We, never- 
theless, take care not to mix up both levels and provide 
a formally clean specification platform in terms of the 
actor model of computation (Agha & Hewitt, 1987). 
In this model each object (actor) constitutes a process 
on its own. Actors communicate by sending messages, 
usually, in an asynchronous mode. Upon reception of 
a message, the receiving actor processes the associ- 
ated method, a program composed of several gram- 
matical predicates, e.g., SYNTAXCheck, which ac- 
counts for morphosyntactic or word order constraints, 
or CONCEPTCheck, which refers to the terminolog- 
ical knowledge representation layer and accounts for 
type and further conceptual admissibility constraints 
(number restrictions, etc.). 

The grammatical description of single words is or- 
ganized in a hierarchy of so-called word actors which 
not only inherit the declarative portions of grammati- 
cal knowledge, but are also supplied with lexicalized 
procedural knowledge that specifies their parsing be- 
havior in terms of a message protocol. A specialized 
actor type, called phrase actor, comprises word actors 
which are connected by dependency relations and en- 
capsulates information about that phrase. 

3.2 Parallelism in Parsing 

In the following, we discuss three stages of increas- 
ing restrictions of parallelism at the word level, all of 
which were considered for the design of the algorithm 
provided in Section 3.3. 

Unbounded Parallelism. Brute-force parsing mod- 
els such as the primordial soup algorithm (Janssen 
et al., 1992), at first sight, exhibit a vast potential for 
parallel execution, since the central operation of build- 
ing a structure from two independent parts (in our ap- 
plication, e.g., the combination of a head and a sin- 
gle modifier) apparently does not require any central- 
ized control. In such an entirely unconstrained paral- 
lel model, a word actor is instantiated from the input 
string and sends search messages to all other word ac- 
tors in order to establish a dependency relation, even- 
tually generating a complete parse of the input. 

Consider, however, the case in which a Noun is pre- 
ceded by a Determiner and an Adjective. In order to 
form the noun phrase [Det Ad j [N]] two computation 
sequences will occur in a primordial soup parser: at- 
taching Det to the N first, then adding Adj, or vice 
versa. Hence, the major drawback of unrestricted 
parallel algorithms is their non-confluency and, sub- 
sequently, either the large (exponential, in the worst 
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Abstract 

We present an approach to parallel natural 
language parsing which is based on a con- 
current, object-oriented model of computa- 
tion. A depth-first, yet incomplete pars- 
ing algorithm for a dependency grammar is 
specified and several restrictions on the de- 
gree of its parallelization are discussed. 

1 Introduction 

There are several arguments why computational lin- 
guists feel attracted by the appeal of parallelism for 
natural language understanding (for a survey, cf. Hahn 
& Adriaens (1994)): the ubiquitous requirement of en- 
hanced efficiency of implementations, its inherent po- 
tential for fault tolerance and robustness, and a flavor 
of cognitive plausibility based on psycholinguistic ev- 
idences from the architecture of the human language 
processor. Among the drawbacks of parallel process- 
ing one recognizes the danger of greedy resource de- 
mands and conomunication overhead for processors 
running in parallel as well as the immense complexity 
of control flow making it hard for humans to properly 
design and debug parallel programs. 

In this paper, we will consider a framework for par- 
allel natural language parsing which summarizes the 
experiences we have made during the development of 
a concurrent, object-oriented parser. We started out 
with a rather liberal conception which allowed for al- 
most unconstrained parallelism. As our work pro- 
gressed, however, we felt the growing need for restrict- 
ing its scope as a continuous "domestication process". 
While still keeping the benefits of parallelism, we have 
arrived at a point where we argue for a basically se- 
rial model patched with several parallel phases rather 
than a basically parallel model with few synchroniza- 
tion checkpoints. Primarily, this change in perspec- 
tive was due to large amounts of artificial ambiguities 
that could be traced to "blind" parallel computations 
with excessive resource allocation requirements. Con- 
tinuously taming the parallel activities of the parser 



and, furthermore, sacrificing highly esteemed theoret- 
ical principles such as the completeness of the parser, 
i.e., the guaranteed production of all analyses for a 
given input, led us to determine those critical portions 
of the parsing process which can reasonably be pur- 
sued in a parallel manner and thus give real benefits in 
terms of efficiency and effectiveness. 

2 Design Requirements for the Parser 

The application framework for the parsing device un- 
der consideration is set up by the analysis of real- 
world expository texts (viz.- information technology 
product reviews and medical findings reports). The 
parser operates as the NLP component of a text knowl- 
edge acquisition and knowledge base synthesis sys- 
tem. 

The analysis of texts (as opposed to sentences in 
isolation) requires the consideration of discourse phe- 
nomena. This includes the interaction of discourse en- 
tities (organized in focus spaces and center lists) with 
structural descriptions from the parser and conceptual 
information from the domain knowledge base. Thus, 
different knowledge sources have to be integrated in 
the course of an incremental text understanding pro- 
cess. 

Within realistic NLP scenarios the parsing device 
will encounter ungrammatical and extragrammatical 
input. In any of these cases, the parser should guar- 
antee a robust, graceful degradation performance, i.e., 
produce fragmentary parses and interpretations corre- 
sponding to the degree of violation or lack of grammar 
constraints. Depending on the severity of fragmenta- 
tion, changes in the parsing strategies which drive the 
text analysis might also be reasonable. 

These requirements obviously put a massive bur- 
den on the control mechanisms of a text understanding 
system. In particular, entirely serial control schemata 
seem inappropriate, since they would introduce arti- 
ficial serialization constraints into basically parallel 
processes (Waltz & Pollack, 1985). 



